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ABSTRACT 

If American schools are to be held accountable for 
pupil achievement, a reliable means of determining the difference 
between a child's actual achievement and his learning potential roust 
be found. This study examines the utilization of composite meas^iCes 
of a pupil's base level of performance as effective predictors: of 
learning expectancy. In addition to an individual student's knowledge 
and skills at learnini commencement, the study also takes into 
account several important factors (such as mental age, years in 
school, age norms, and school history} to provide a statistically 
significant performance potential for each child. In this way, the 
approach remains sensitive to the goals of specific programs or the 
objectives of a particular school district. The data are also 
comparable across teachers, and cc^ild possibly be ujsad as tha basis 
for a teacher evaluation system. (Author) 



FILMED FROM BEST AVAILABLE COPY 



OS 06PAHTMEN1 OF HEALTH 
EDUCATION a WELFARE 
NATIONAL INSTITUTE OF 
EDUCATION 

Db-cen exAC^tv AS »E:f-.-ED t eov 

AT'SO " PO^N^S Ot- V'E A Oe OP'N'ONS 
blAlFODONO^ NFCF^'-APav (?£PRE 
SEN'" Of <^ 'CiAL SATiOSA. .NS^tUiTE 0» 

AN ACCOUNTABILITY APPROACH US ING EXPECTANCY CRITERIA 

by 

Thomas H . Metos 
Elanny T, Luty 

If American public schools are to be he'id accountable for 
pupil achievement, we must find a reliable means of determining 
the difference between the child's actual achie^vement and his 
learning potential. This study examines the utilization of 
composite measures of a pupil's base level of performance as 
effective predictors of learning expectancy. The value o' such 
prognostication is multitudinous. For example, if one \:cre able 
to determine whether sufficient discrepancy existed ::»etween a 
pupil's actual performance and his expected performance, ciagnos 
could promptly begin to determine the cause of the discrepuncy 
in order to prescribe treatment. Ultimately, it is conceivable 
to envision compos i te 1 earni ng expectancy 1 evel s servi nc us re - 
I ferents to determine, in terms of expectancy, wl^ether sufficient 
learning has occured in a classroom, school; or district. Since 
the evaluation of performance would not be based on a norm group 
but rather on individual learning expectancy levels, the data 
becomes comparable across teachers and possibly could be used 
as a basis for teacher evaluation system- 
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The many factors involved in contributing to a pupil's actua,] 
learning performance have made it extremely difficult to utilize 
measures which are relevant and objective. The problem becomes 
additionally complex if one attempts to determine whether a 
pupil is performing at his "expected*' \evel of performance. 
Measures of actual learning performance, therefore, nececbitate 
not only the measurement of achievement in its quantitative sense 
but also the measurement of achi e vemen t'-rel ated variables. Mea- 
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surement of such variables, however, necessi tates the use of 
multiple input factors which in some way embody these achievement- 
related criteria and in addition have the virtue of meas ureabi 1 i ty . 
The findings of the authors suggests that the learning expectancy 
level formula exemplifies such criteria. 

METHODS AND PROCEDURES 
Different methods for determining a pupil *s learning potential, 
i.e., his learning expectancy level (LEL) ha^'e been suggested. 
These include the following: 

1. LEL l...Kaluger and Kolson 

2. LEL 2. . .Bond and Tinker 

3. LEL 3.. .Harris - Expectancy Quotient 

4. LEL 4. ..Harris - Reading Quotient 

5. LEL 5. ..Index of Learning Potential 

6. LEL 6...Durrell and Brassard 

The investigators attempted to determin^ the validity of using 
learning expectancy level (LEL) o.riteria at the third and sixth 
grade levels for reading and mathematics as they relate to pupil 
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performance. By applying the above learning expectancy level 
criteria simultaneously to an individual, a single criterion 
measure of the highest capability of prediction could be de- 
termined. This could be evidenced by relating actual student 
achievement with the various learning expectancy level criteria. 
For the purpose of the i n ves t i ga t i or , the question became: 

1 ) What di f fere nee s exist between the proposed 
learni ng expectancy leve 1 cri teri a? 

2) Which of the existing criteria correlate most 
closely with actual student achievement* 

If such criteria were deemed feasible in terms of their validity, 
then additional reliability procedures of each criterion could be 
made insofar as these criteria relate toward predicting pupil 
performance , 

Two elementary schools were selected from a population of 
eighteen elementary schools in the Mesa Public Schools District, 
Mesa, Arizona. The two schools were selected by means of random 
selection drawing procedures, the third and sixth grade within 
each school bei ng ful ly tested . In thi s manner , the data re- 
mained in accord with cluster sampling requirements since there 
was a random sample of clusters, each cluster being fully tested. 
As such the bias-free virtue of random selection applied to the 
sample of larger units, viz., schools. 

Pupil data used for inclusion into the various learning 
expectancy level formulae was collected by means of the follow- 
ing instruments: Me tropol i tan Ma th Subtest, Ga tes -MacGi n i ti e 
Reading Test, Metropolitan Achievement Test, Otis-Lennon Mental 



Ability Test, Harcourt's Analysis of Learning Potential, and 

the Durrel ] Reading Test, 

In order to equate the diverse LEL criteria mathematically, 

each LEL was converted to a "z" score prior to data analysis. 

Treatment of the data was as follows: 

!• An analysis of Vc\riance was performed in order to 
determine whether there existed any significant 
differences in pupH's reading and mathematics scores 
from the time they entered school to when they completed 
school • 

2. An analysis of variance was performed in order to 
determine whether there existed any differences in the 
predictive value of the various learning expectancy 
level cri teri a . 

3, A canonical correi^ttion was performed using post 
reading and mathenatics scores and the six learning 
expectancy level criteria in order to determine which 
LEL predictor estimate correlated most closely with 
final achievement in reading and mathematics. 

According to Cooley and Lohnes ^he utilization of the canonical 
c.orrelaticn allows an ove/all test of relation:, between sets of 
vari abl es • ^ As such, it pe rmi ts the sets to be combi ned in all 
possible ways to reveal maximum correlation between components. 
The canonical correlation was used, therefore, to test the relation- 
ship between the following two sets of variables: 
LEARNING EXPECTANCY LEVEL VARIABLES - PREDICTOR CORRELATES 

1. LEL l,,,Kaluger and Kolson 
4 
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INDIVIDUAL ABILITY VARIABLES - CRITERION MEASURES 
1 . Matheme.ti cs 

2. Reading 

3. I. Q. 

Kendall pointed out, through the canonical correlation 

method, one may reduce the relationships between the two sets 
8 

of Vciriables. In this study, the canonical correlation involved 
multiple predictors and multiple correlations. With each pair of 
canonical variates maximally correlated, the canonical correlation 
method revealed the degree of correlation underlying the two sets 
i.e., the maximum correlation between linear functions of nine 
two-dimensional sets of variables. 

FINDINGS 

Table 1 presents the variance analysis used to ascertain .he 
statistical significance of the differences among the pre-test 
to post-test gains. The analysis revealed a significant "F" ratio 
for reading at the third grade of 106,14 and at the sixth grade 
of 24.39. Analysis for mathematics revealed a significant "F" 
ratio at the third grade of 17.43, and at the sixth grade of 34.83. 
The results of the analysis indicated there existed significant 
differences from pre-tesc to post-test in pupi 1 ' s. read' ng and 
mathematics ability at both the third and sixth grade levels. 
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Table 2 presents the variance analysis used to ascertain 
the statistical significance of the differences among the six 
learning expectancy level criteria at the third and sixth grade 
levels A significant "F'' ratio of 16.95 was established for 
the six learning expectancy level criteria at the third grade. 
At the sixth grade, the "F" ratio of 3.71 was also found to be 
significant. The statistically significant "F" ratio revealed 
that there existed significant differences in the predictive 
values of the various learning expectancy level criteria in- 
dicating that the six LEL formulae were not equal in predictive 
capabi 1 i ty . 

Table 3, X of Successive Latent Roots presenting the 

comprehensive canonical results, i.e., R = .99, A 

c . max . ^ ^ 

.00016 = 3091.84, NDF = 18, p <.05 Indicates a significant 
overall relationship between the predictor correlates, I.e., the 
LEL's and the criterion variables. I.e., mathematics, reading and 
I.Q. 

Table 4 presents the overall correlation matrix R']2 The 
significant correlation relating reading and an LEL were with LEL 2 
(.81) and LEL 1 (.80). The correlation values of either .81 or .80 
have indicated that either LEL 2 LEL 1 may be used to predict, 
wi th statistical si gni f i cdiice , 1 3 arn i ng expectancy in readi ng 
performance. The significant correlations between mathematics 
and an LEL were with LEL 1 (.80) and LEL 2 (.79). The correlation 
values of either .80 or .79 have indicated that either LEL 1 or LEL 
2 may be used to predict, with statistical signific.n j, learning 
expectancy in mathematics performance. 
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SUfn^ARY AND DISCUSSION 
It was concluded that prediction of reading and mathematics 
expectancy can be statistically significant with a specific LEL 
criterion, viz., LEL 1 or LEL 2. Expectancy levels so de rived 
from empirical data meet the requirement of comparability of 
actiievement and assessment measures since the two are based 
virtually on the same group. It is important however, to ex- 
ercise considerable caution in using the LEL ^ince: 

1. The achievement score v/hich is compared with the 
"expected" value has virtually the same standard 
error o*^ measurement. 

2. The predicted achievement level is subject to an 
error of estimate since the basic correlate is less 
than perfect, 

3. The stability of a detern-ined learning expectancy 
value remains, as yet, unkriown, 

4. Validity is not a general i zab 1 e phenomenon and might 
vary from group to group. Therefore, it would need 
verification in each study, and certainly in each 
study which used as subjects a group different in 
some si gni fi cant respect from the sample uti 1 1 zed 

in this study. 

Therefore, use of the^ LEL as a sole criterion of expectancy 
without subsequent verification would be unwarranted. Appropri- 
ately used, however, learning expectancy criteria can simplify 
the task of the teacher in determining whether or not a student 
is actually working to his "expectancy''. 
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Being held responsible for educational outcomes, i.e., being 
held accountable, necessitates reasonable estimates of contribu- 
tion to pupil performance by individual agents in the educational 
process. Estimates of such contribution could be readily inferred 
from comparative analysis of the Lr.L in different classrooms, 
schools, and districts, i.e. whether pupils tend to fall above or 
below the expected performance. As such, relative evaluative 
criteria could be established from an analysis of results obtained 
by personnel working in comparable circumstances with comparable 
variables. Such analysis would make it possible to determine the 
extent to which measured teacher characteristics are significantly 
correlated with tec;cher cf f c cti venes s . Potentially, this infoniia- 
tier could have important policy implications and impact on school 
management, resource allocation and personnel practices* 

Responsibility for educational outcomes, i.e. accountability 
cannot be fully realized unless we can identify the matches and 
mis-matches between the proposed (expected) performance level 
and the actual performance level. Only then can possible solution 
strategies bo Implemented. The main advantage of the approach 
described herein is that it takes into account individual students 
knowledge and skills upon commencement of learning* In addition 
it takes into account several important factors (such as mental age, 
years in school , age norms , school history) in order to provide 
a statistically significant performance potential for each individual 
child. As such, the approach remains sensitive to the goals of 
speci f i c programs and/or certain objectives cons ide red to be 
especially important to a given school district. 

ER?C 8 



In summary, the rrethoJology described herein has the 
potential of serving as an accountability system base, utilizing 
a quality ass' ance system of data that is relevant, objective 
and readily assessable. 
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